Research Article | Open Access

Volume 2022 |Article ID 0005 | https://doi.org/10.34133/plantphenomics.0005

SE-COTR: A Novel Fruit Segmentation Model for Green Apples Application in Complex Orchard

Zhifen Wang,¹ Zhonghua Zhang,¹ Yuqi Lu,¹ Rong Luo,² Yi Niu,³ Xinbo Yang,¹ Shaoxue Jing,⁴ Chengzhi Ruan,^5,6 Yuanjie Zheng,¹ Weikuan Jia ^1,6

¹School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China
²State Key Laboratory of Biobased Materials and Green Papermaking, Qilu University of Technology (Shandong Academy of Science), Jinan 25035, China
³School of Informatics, University of Leicester, Leicester LE1 7RH, UK
⁴Department of Engineering Design and Mathematics, University of the West of England, Bristol BS16 1QY, UK
⁵Fujian Key Laboratory of Intelligent Control and Manufacturing of Agricultural Machinery, Wuyishan 354300, China
⁶Key Laboratory of Facility Agriculture Measurement and Control Technology and Equipment of Machinery Industry, Jiangsu University, Zhenjiang 212013, China

Abstract

Because of the unstructured characteristics of natural orchards, the efficient detection and segmentation applications of green fruits remain an essential challenge for intelligent agriculture. Therefore, an innovative fruit segmentation method based on deep learning, termed SE-COTR (segmentation based on coordinate transformer), is proposed to achieve accurate and real-time segmentation of green apples. The lightweight network MobileNetV2 is used as the backbone, combined with the constructed coordinate attention-based coordinate transformer module to enhance the focus on effective features. In addition, joint pyramid upsampling module is optimized for integrating multiscale features, making the model suitable for the detection and segmentation of target fruits with different sizes. Finally, in combination with the outputs of the function heads, the dynamic convolution operation is applied to predict the instance mask. In complex orchard environment with variable conditions, SE-COTR achieves a mean average precision of 61.6% with low complexity for green apple fruit segmentation at severe occlusion and different fruit scales. Especially, the segmentation accuracy for small target fruits reaches 43.3%, which is obviously better than other advanced segmentation models and realizes good recognition results. The proposed method effectively solves the problem of low accuracy and overly complex fruit segmentation models with the same color as the background and can be built in portable mobile devices to undertake accurate and efficient agricultural works in complex orchard.